User interface apparatus for vehicle, and vehicle

ABSTRACT

A user interface apparatus configured to be installed in a vehicle, the apparatus includes a touch screen; a gesture detecting unit configured to detect a gesture of a user and convert the gesture into an electrical signal; and at least one processor configured to determine whether a criteria for switching to an autonomous driving state is satisfied in response to the gesture detected by the gesture detecting unit being applied from a driver seat of the vehicle to the touch screen, further, in response to the criteria for switching to the autonomous driving state being satisfied, the at least one processor is further configured to switch a state of the vehicle to the autonomous driving state.

CROSS-REFERENCE TO RELATED APPLICATION

This application is the Continuation of U.S. application Ser. No.15/429,843, filed on Feb. 10, 2017, which claims the priority benefit ofU.S. Provisional Application No. 62/294,503, filed on Feb. 12, 2016, allof which are hereby expressly incorporated by reference into the presentapplication.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a user interface apparatus for vehicle,and a vehicle including the same.

2. Description of the Related Art

A vehicle is an apparatus that moves in a direction desired by a userriding therein. A representative example of a vehicle is an automobile.A variety of sensors and electronic devices are mounted in vehicles forconvenience of a user who uses the vehicle. In particular, for drivingconvenience, an Advanced Driver Assistance System (ADAS) has beenactively studied. In addition, enormous efforts have been being made todevelop autonomous vehicles.

In addition, various user interface apparatuses are provided in avehicle. A user interface apparatus for a vehicle may be called a CenterInformation Display (CID) and an Audio Video navigation (AVN) system.Such a user interface apparatus receives a user input and outputsinformation for a user.

However, the user interface apparatus can be sometimes a hindrance todriving a vehicle. For example, if a user, applies a touch input orstares at a display unit to see information while driving the car, theuser cannot focus on driving and may cause an accident.

SUMMARY OF THE INVENTION

The present invention has been made in view of the above problems, andit is one object of the present invention to provide a user interfaceapparatus for a vehicle, which determines whether a gesture is appliedfrom the driver seat or the front passenger seat, and operatesdifferently based on the determination.

It is another object of the present invention to provide a vehicleincluding a user interface apparatus.

In accordance with an embodiment of the present invention, the above andother objects can be accomplished by the provision of an user interfaceapparatus for a vehicle, the apparatus including a display unit, agesture input unit configured to detect a gesture of a user, and aprocessor configured to, based on a motion trajectory of the gesture,determine a direction from which the gesture is applied, and, based onwhether the gesture is applied from a driver seat or a front passengerseat, control the display unit to display a preset different screen.

Further scope of applicability of the present invention will becomeapparent from the detailed description given hereinafter. However, thedetailed description and specific examples, while indicating preferredembodiments of the invention, are given by illustration only, sincevarious changes and modifications within the spirit and scope of theinvention will become apparent to those skilled in the art from thisdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments will be described in detail with reference to thefollowing drawings in which like reference numerals refer to likeelements wherein:

FIG. 1 is a diagram illustrating the external appearance of a vehicleaccording to an embodiment of the present invention;

FIG. 2 are different angled views of the external appearance of avehicle according to an embodiment of the present invention;

FIGS. 3 and 4 are diagrams illustrating the interior configuration of avehicle according to an embodiment of the present invention;

FIGS. 5 and 6 are diagrams illustrating an object according to anembodiment of the present invention;

FIG. 7 is a block diagram illustrating a vehicle according to anembodiment of the present invention;

FIG. 8 is a block diagram illustrating a user interface apparatus for avehicle according to an embodiment of the present invention;

FIG. 9 is a flowchart illustrating how a user interface apparatus for avehicle operates according to an embodiment of the present invention;

FIGS. 10 to 12 are diagrams illustrating how a user interface apparatusfor a vehicle operates according to a gesture input according to anembodiment of the present invention;

FIG. 13 is a flowchart illustrating an operation of a user interfaceapparatus for a vehicle according to an embodiment of the presentinvention;

FIGS. 14 to 17 are diagrams illustrating how a user interface apparatusexecutes an application when displaying of a screen is disabled,according to an embodiment of the present invention;

FIG. 18 is a diagram illustrating how an application corresponding to astate of a vehicle is executed, according to an embodiment of thepresent invention;

FIG. 19 is a flowchart illustrating how a user interface apparatus for avehicle operates according to an embodiment of the present invention;

FIG. 20 is a diagram illustrating an example of how a user interfaceapparatus for a vehicle operates based on driver gaze informationaccording to an embodiment of the present invention;

FIG. 21 is a flowchart illustrating how a user interface apparatus for avehicle operates according to an embodiment of the present invention;

FIGS. 22 to 24 are diagram illustrating examples of the operationdescribed with reference to FIG. 21;

FIGS. 25A to 27 are diagrams illustrating various examples of how a userinterface apparatus operates based on a gaze of a user according to anembodiment of the present invention;

FIG. 28 is a diagram illustrating how a user interface apparatus for avehicle, including a plurality of display units, operates according toan embodiment of the present invention;

FIG. 29 is a diagram illustrating how a user interface apparatus for avehicle, including a plurality of display units, operates according toan embodiment of the present invention;

FIG. 30 is a diagram illustrating how a user interface apparatus for avehicle used by multiple users operates according to an embodiment ofthe present invention;

FIGS. 31 to 33 are diagrams illustrating how to switch to an autonomousdriving state by a user interface apparatus for a vehicle according toan embodiment of the present invention; and

FIGS. 34 to 36 are diagrams illustrating how to operate after switchingto an autonomous driving state according to an embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, the embodiments disclosed in the present specification willbe described in detail with reference to the accompanying drawings, andthe same or similar elements are denoted by the same reference numeralseven though they are depicted in different drawings and redundantdescriptions thereof will be omitted. In the following description, withrespect to constituent elements used in the following description, thesuffixes “module” and “unit” are used or combined with each other onlyin consideration of ease in the preparation of the specification, and donot have or serve as different meanings. Accordingly, the suffixes“module” and “unit” may be interchanged with each other. In addition,the accompanying drawings are provided only for a better understandingof the embodiments disclosed in the present specification and are notintended to limit the technical ideas disclosed in the presentspecification. Therefore, it should be understood that the accompanyingdrawings include all modifications, equivalents and substitutionsincluded in the scope and sprit of the present invention.

Although the terms “first,” “second,” etc., may be used herein todescribe various components, these components should not be limited bythese terms. These terms are only used to distinguish one component fromanother component. When a component is referred to as being “connectedto” or “coupled to” another component, it may be directly connected toor coupled to another component or intervening components may bepresent. In contrast, when a component is referred to as being “directlyconnected to” or “directly coupled to” another component, there are nointervening components present.

As used herein, the singular form is intended to include the pluralforms as well, unless the context clearly indicates otherwise. In thepresent application, it will be further understood that the terms“comprises”, includes,” etc. specify the presence of stated features,integers, steps, operations, elements, components, or combinationsthereof, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components, orcombinations thereof.

A vehicle as described in this specification may include an automobileand a motorcycle. Hereinafter, a description will be given based on anautomobile. A vehicle as described in this specification may include allof an internal combustion engine vehicle including an engine as a powersource, a hybrid vehicle including both an engine and an electric motoras a power source, and an electric vehicle including an electric motoras a power source. In the following description, “the left side of thevehicle” refers to the left side in the forward driving direction of thevehicle, and “the right side of the vehicle” refers to the right side inthe forward driving direction of the vehicle.

FIGS. 1 to 7 illustrate a vehicle 100 including a plurality of wheelsrotated by a power source, and a steering input apparatus 510 forcontrolling a driving direction of the vehicle 100. The vehicle 100 maybe an autonomous vehicle.

Further, the vehicle 100 can be switched to an autonomous driving modeor a manual mode in response to a user input. For example, in responseto a user input received through a user interface apparatus 200, thevehicle 100 can be switched from a manual mode to an autonomous drivingmode, or vice versa. The vehicle 100 may also be switched to anautonomous mode or a manual mode based on driving situation information.

The driving situation information may include information on an objectoutside the vehicle 100, navigation information, and vehicle stateinformation. For example, the vehicle 100 can be switched from themanual mode to the autonomous driving mode, or vice versa, based ondriving environment information generated by the object detection device300.

In another example, the vehicle 100 can be switched from the manual modeto the autonomous driving mode, or vice versa, based on drivingenvironment information received through a communication device 400. Thevehicle 100 can be switched from the manual mode to the autonomousdriving mode, or vice versa, based on information, data, and a signalprovided from an external device.

When the vehicle 100 operates in the autonomous driving mode, theautonomous vehicle 100 can operate based on an operation system 700. Forexample, the autonomous vehicle 100 can operate based on information,data, or signals generated by a driving system 710, a parking system740, and a parking out system 750.

While operating in the manual mode, the autonomous vehicle 100 mayreceive a user input for driving of the vehicle 100 through a drivingmanipulation device 500. In response to the user input received throughthe driving manipulation device 500, the vehicle 100 can operate.

The term “overall length” means the length from the front end to therear end of the vehicle 100, the term “overall width” means the width ofthe vehicle 100, and the term “overall height” means the height from thebottom of the wheel to the roof. In the following description, the term“overall length direction L” may mean the reference direction for themeasurement of the overall length of the vehicle 100, the term “overallwidth direction W” may mean the reference direction for the measurementof the overall width of the vehicle 100, and the term “overall heightdirection H” may mean the reference direction for the measurement of theoverall height of the vehicle 100.

As illustrated in FIG. 7, the vehicle 100 may include the user interfaceapparatus 200, the object detection device 300, the communication device400, the driving manipulation device 500, a vehicle drive device 600,the operation system 700, a navigation system 770, a sensing unit 120,an interface 130, a memory 140, a controller 170, and a power supplyunit 190. The vehicle 100 may further include other components inaddition to the aforementioned components, or may not include some ofthe aforementioned components.

The user interface apparatus 200 is provided to support communicationbetween the vehicle 100 and a user. The user interface apparatus 200 canreceive a user input, and provide information generated in the vehicle100 to the user. The vehicle 100 may enable User Interfaces (UI) or UserExperience (UX) through the user interface apparatus 200.

The user interface apparatus 200 may include an input unit 210, aninternal camera 220, a biometric sensing unit 230, an output unit 250,and a processor 270. The user interface apparatus 200 may furtherinclude other components in addition to the aforementioned components,or may not include some of the aforementioned components.

The input unit 210 is configured to receive information from a user, anddata collected in the input unit 210 can be analyzed by the processor270 and then processed into a control command of the user. The inputunit 210 can be disposed inside the vehicle 100. For example, the inputunit 210 can be disposed in a region of a steering wheel, a region of aninstrument panel, a region of a seat, a region of each pillar, a regionof a door, a region of a center console, a region of a head lining, aregion of a sun visor, a region of a windshield, or a region of awindow.

The input unit 210 may include a voice input unit 211, a gesture inputunit 212, a touch input unit 213, and a mechanical input unit 214. Thevoice input unit 211 may convert a voice input of a user into anelectrical signal. The converted electrical signal may be provided tothe processor 270 or the controller 170. The voice input unit 211 mayinclude one or more microphones.

The gesture input unit 212 may convert a gesture input of a user into anelectrical signal. The converted electrical signal may be provided tothe processor 270 or the controller 170. The gesture input unit 212 candetect a gesture of a user and include at least one of an infraredsensor and an image sensor for sensing a gesture input of a user.

The gesture input unit 212 may sense a three-dimensional (3D) gestureinput of a user. Thus, the gesture input unit 212 may include aplurality of light emitting units for outputting infrared light, or aplurality of image sensors. The gesture input unit 212 can sense the 3Dgesture input by employing a Time of Flight (TOF) scheme, a structuredlight scheme, or a disparity scheme.

The touch input unit 213 can convert a user's touch input into anelectrical signal, and the converted electrical signal can be providedto the processor 270 or the controller 170. The touch input unit 213 mayinclude a touch sensor for sensing a touch input of a user. The touchinput unit 210 may be integrally formed with a display unit 251 toimplement a touch screen, and the touch screen can provide an inputinterface and an output interface between the vehicle 100 and the user.

The mechanical input unit 214 may include at least one of a button, adome switch, a jog wheel, and a jog switch. An electrical signalgenerated by the mechanical input unit 214 may be provided to theprocessor 270 or the controller 170. The mechanical input unit 214 canbe located on a steering wheel, a center fascia, a center console, acockpit module, a door, etc.

Further, the internal camera 220 can acquire images of the inside of thevehicle 100. Also, the processor 270 can sense a user's condition basedon the images of the inside of the vehicle 100. The processor 270 canacquire information on an eye gaze of the user and can sense a gestureof the user from the images of the inside of the vehicle 100.

The biometric sensing unit 230 can acquire biometric information of theuser. The biometric sensing unit 230 may include a sensor for acquirebiometric information of the user, and may utilize the sensor to acquirefinger print information, heart rate information, etc. of the user. Thebiometric information may be used for user authentication.

The output unit 250 is configured to generate a visual, audio, ortactile output. The output unit 250 may include at least one of adisplay unit 251, a sound output unit 252, and a haptic output unit 253.The display unit 251 may display graphic objects corresponding tovarious types of information. The display unit 251 may include at leastone of a Liquid Crystal Display (LCD), a Thin Film Transistor-LiquidCrystal Display (TFT LCD), an Organic Light-Emitting Diode (OLED), aflexible display, a 3D display, and an e-ink display.

The display unit 251 may form an inter-layer structure together with thetouch input unit 213, or may be integrally formed with the touch inputunit 213 to implement a touch screen. The display unit 251 may also beimplemented as a Head Up Display (HUD). When implemented as a HUD, thedisplay unit 251 may include a projector module in order to outputinformation through an image projected on a windshield or a window.

The display unit 251 may include a transparent display attached on thewindshield or the window. The transparent display can display apredetermined screen with a predetermined transparency. In order toachieve the transparency, the transparent display may include at leastone of a transparent Thin Film Electroluminescent (TFEL) display, anOrganic Light Emitting Diode (OLED) display, a transparent LiquidCrystal Display (LCD), a transmissive transparent display, and atransparent Light Emitting Diode (LED) display. The transparency of thetransparent display may be adjustable.

In addition, the user interface apparatus 200 may include a plurality ofdisplay units 251 a to 251 g. The display unit 251 can be disposed in aregion of a steering wheel, a region 251 a, 251 b, or 251 e of aninstrument panel, a region 251 d of a seat, a region 251 f of eachpillar, a region 251 g of a door, a region of a center console, a regionof a head lining, a region of a sun visor, a region 251 c of awindshield, or a region 251 h of a window.

The sound output unit 252 converts an electrical signal from theprocessor 270 or the controller 170 into an audio signal, and outputsthe audio signal. Thus, the sound output unit 252 may include one ormore speakers. In addition, the haptic output unit 253 generates atactile output. For example, the haptic output unit 253 can operate tovibrate a steering wheel, a safety belt, and seats 110FL, 110FR, 110RL,and 110RR so as to allow a user to recognize the output.

Further, the processor 270 can control the overall operation of eachunit of the user interface apparatus 200. The user interface apparatus200 may include a plurality of processors 270 or may not include theprocessor 270. When the user interface apparatus 200 does not includethe processor 270, the user interface apparatus 200 can operate underthe control of the controller 170 or a processor of a different deviceinside the vehicle 100. In addition, the user interface apparatus 200may be referred to as a display device for vehicle and can operate underthe control of the controller 170.

The object detection device 300 is an apparatus for detecting an objectlocated outside the vehicle 100 and can generate object informationbased on sensing data. The object information may include information asto existence of an object, location information of an object,information on a distance between the vehicle 10 and the object, andinformation on relative speed of the vehicle 100 and the object.Further, the object may be any of various objects related to travellingof the vehicle 100.

Referring to FIGS. 5 and 6, an object o may include a lane OB10, anearby vehicle OB11, a pedestrian OB12, a two-wheeled vehicle OB13, atraffic signal OB14 and OB15, a light, a road, a structure, a bump, ageographical feature, an animal, etc. Further, the lane OB10 may be alane in which the vehicle 100 is traveling or a lane next to the lane inwhich the vehicle 100 is traveling and may include left and right linesthat define the lane.

The nearby vehicle OB11 may be a vehicle that is travelling in thevicinity of the vehicle 100. Further, the nearby vehicle OB11 may be avehicle within a predetermined distance from the vehicle 100. Forexample, the nearby vehicle OB11 may be a vehicle that is preceding orfollowing the vehicle 100. In addition, the pedestrian OB12 may be aperson in the vicinity of the vehicle 100 such as a person within apredetermined distance from the vehicle 100. For example, the pedestrianOB12 may be a person on a sidewalk or on the roadway.

The two-wheeled vehicle OB13 is a vehicle that is located in thevicinity of the vehicle 100 and moves with two wheels. The two-wheeledvehicle OB13 may be a vehicle that has two wheels within a predetermineddistance from the vehicle 100. For example, the two-wheeled vehicle OB13may be a motorcycle or a bike on a sidewalk or the roadway.

The traffic signal may include a traffic light OB15, a traffic signplate OB14, and a pattern or text painted on a road surface. The lightmay be light generated by a lamp provided in the nearby vehicle.Further, the light may be light generated by a street light. The lightmay also be solar light. The road may include a road surface, a curve,and slopes, such as an upward slope and a downward slope.

The structure may be a body located around the road in the state ofbeing fixed onto the ground. For example, the structure may include astreetlight, a roadside tree, a building, a traffic light, and a bridge.The geographical feature may include a mountain and a hill. In addition,the object may be classified as a movable object or a stationary object,and the movable object may include a nearby vehicle and a pedestrian.For example, the stationary object may include a traffic signal, a road,and a structure.

The object detection device 300 may include a camera 310, a radar 320, alidar 330, an ultrasonic sensor 340, an infrared sensor 350, and aprocessor 370. The object detection device 300 may further include othercomponents in addition to the aforementioned components, or may notinclude some of the aforementioned components.

The camera 310 can be located at an appropriate position outside thevehicle 100 in order to acquire images of the outside of the vehicle100. The camera 310 may be a mono camera, a stereo camera 310 a, anAround View Monitoring (AVM) camera 310 b, or a 360-degree camera.Further, the camera 310 can acquire location information of an object,information on a distance to the object, and information on a speedrelative to the object, by using various image processing algorithms.

For example, the camera 310 can acquire the information on the distanceto the object and information on the speed relative to the object, basedon change over time in size of the object, the change which is detectedin acquired images. In another example, the camera 310 can acquire theinformation on the distance to the object and information on the speedrelative to the object, by using a pin hole model or profiling a roadsurface.

In addition, the camera 310 can acquire the information on the distanceto the object and the information on the speed relative to the object,based on information on disparity in stereo images acquired by a stereocamera 310 a. For example, the camera 310 can be disposed near a frontwindshield in the vehicle 100 in order to acquire images of the front ofthe vehicle 100. Alternatively, the camera 310 can be disposed around afront bumper or a radiator grill.

In another example, the camera 310 can be disposed near a rear glass inthe vehicle 100 in order to acquire images of the rear of the vehicle100. Alternatively, the camera 310 can be disposed around a rear bumper,a trunk, or a tailgate. In yet another example, the camera 310 can bedisposed near at least one of the side windows in the vehicle 100 inorder to acquire images of the side of the vehicle 100. Alternatively,the camera 310 can be disposed around a side mirror, a fender, or adoor. The camera 310 can provide an acquired image to the processor 370.

The radar 320 may include an electromagnetic wave transmission unit andan electromagnetic wave reception unit. The radar 320 may be realized asa pulse radar or a continuous wave radar depending on the principle ofemission of an electronic wave. In addition, the radar 320 may berealized as a Frequency Modulated Continuous Wave (FMCW) type radar or aFrequency Shift Keying (FSK) type radar depending on the waveform of asignal.

The radar 320 can detect an object through the medium of anelectromagnetic wave by employing a time of flight (TOF) scheme or aphase-shift scheme, and can detect a location of the detected object,the distance to the detected object, and the speed relative to thedetected object. Further, the radar 320 can be located at an appropriateposition outside the vehicle 100 in order to sense an object located infront of the vehicle 100, an object located to the rear of the vehicle100, or an object located to the side of the vehicle 100.

The lidar 330 may include a laser transmission unit and a laserreception unit. Further, the lidar 330 may be implemented by the TOFscheme or the phase-shift scheme. The lidar 330 may also be implementedas a drive type lidar or a non-drive type lidar. When implemented as thedrive type lidar, the lidar 300 can rotate by a motor and detect anobject in the vicinity of the vehicle 100.

When implemented as the non-drive type lidar, the lidar 300 can utilizea light steering technique to detect an object located within apredetermined distance from the vehicle 100. The lidar 330 can alsodetect an object through the medium of laser light by employing the TOFscheme or the phase-shift scheme, and can detect a location of thedetected object, the distance to the detected object, and the speedrelative to the detected object. Further, the lidar 330 can be locatedat an appropriate position outside the vehicle 100 in order to sense anobject located in front of the vehicle 100, an object located to therear of the vehicle 100, or an object located to the side of the vehicle100.

The ultrasonic sensor 340 may include an ultrasonic wave transmissionunit and an ultrasonic wave reception unit. The ultrasonic sensor 340can detect an object based on an ultrasonic wave, and can detect alocation of the detected object, the distance to the detected object,and the speed relative to the detected object. Further, the ultrasonicsensor 340 can be located at an appropriate position outside the vehicle100 in order to detect an object located in front of the vehicle 100, anobject located to the rear of the vehicle 100, and an object located tothe side of the vehicle 100.

The infrared sensor 350 may include an infrared light transmission unitand an infrared light reception unit. The infrared sensor 350 can detectan object based on infrared light, and can detect a location of thedetected object, the distance to the detected object, and the speedrelative to the detected object. Further, the infrared sensor 350 can belocated at an appropriate position outside the vehicle 100 in order tosense an object located in front of the vehicle 100, an object locatedto the rear of the vehicle 100, or an object located to the side of thevehicle 100.

The processor 370 can control the overall operation of each unit of theobject detection device 300. The processor 370 can detect and classifyan object by comparing data sensed by the camera 310, the radar 320, thelidar 330, the ultrasonic sensor 340, and the infrared sensor 350 withpre-stored data. Further, the processor 370 can detect and track anobject based on acquired images. The processor 370 can, for example,calculate the distance to the object and the speed relative to theobject by using image processing algorithms.

For example, the processor 370 can acquire information on the distanceto the object and information on the speed relative to the object basedon change over time in size of the object in acquired images. Forexample, the processor 370 can acquire information on the distance tothe object or information on the speed relative to the object byemploying a pin hole model or by profiling a road surface. For example,the processor 370 can acquire information on the distance to the objectand information on the speed relative to the object based on informationon disparity in stereo images acquired from the stereo camera 310 a.

The processor 370 can detect and track an object based on a reflectionelectromagnetic wave which is formed as a result of reflection atransmission electromagnetic wave by the object. Based on theelectromagnetic wave, the processor 370 can, for example, calculate thedistance to the object and the speed relative to the object.

The processor 370 can detect and track an object based on a reflectionlaser light which is formed as a result of reflection of transmissionlaser by the object. Based on the laser light, the processor 370 can,for example, calculate the distance to the object and the speed relativeto the object. The processor 370 can detect and track an object based ona reflection ultrasonic wave which is formed as a result of reflectionof a transmission ultrasonic wave by the object. Based on the ultrasonicwave, the processor 370 can, for example, calculate the distance to theobject and the speed relative to the object.

The processor 370 can detect and track an object based on reflectioninfrared light which is formed as a result of reflection of transmissioninfrared light by the object. Based on the infrared light, the processor370 can, for example, calculate the distance to the object and the speedrelative to the object.

The object detection device 300 may include a plurality of processors370 or may not include the processor 370. For example, each of thecamera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, andthe infrared sensor 350 may include its own processor. When the objectdetection device 300 does not include the processor 370, the objectdetection device 300 can operate under the control of the controller 170or a processor inside the vehicle 100. The object detection device 300can operate under the control of the controller 170.

The communication device 400 is configured to perform communication withan external device. Here, the external device may be a nearby vehicle, amobile terminal, or a server. To perform communication, thecommunication device 400 may include at least one of a transmissionantenna, a reception antenna, a Radio Frequency (RF) circuit capable ofimplementing various communication protocols, and an RF device.

The communication device 400 may include a short-range communicationunit 410, a location information unit 420, a V2X communication unit 430,an optical communication unit 440, a broadcasting transmission andreception unit 450, an Intelligent Transport Systems (ITS) communicationunit 460, and a processor 470. The communication device 400 may furtherinclude other components in addition to the aforementioned components,or may not include some of the aforementioned components.

The short-range communication unit 410 is configured to performshort-range communication. The short-range communication unit 410 maysupport short-range communication using at least one of Bluetooth™,Radio Frequency IDdentification (RFID), Infrared Data Association(IrDA), Ultra-WideBand (UWB), ZigBee, Near Field Communication (NFC),Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless USB (WirelessUniversal Serial Bus).

The short-range communication unit 410 may form wireless area networksto perform short-range communication between the vehicle 100 and atleast one external device. The location information unit 420 isconfigured to acquire location information of the vehicle 100. Forexample, the location information unit 420 may include a GlobalPositioning System (GPS) module or a Differential Global PositioningSystem (DGPS) module.

The V2X communication unit 430 is configured to perform wirelesscommunication between a vehicle and a server (that is, vehicle to infra(V2I) communication), wireless communication between a vehicle and anearby vehicle (that is, vehicle to vehicle (V2V) communication), orwireless communication between a vehicle and a pedestrian (that is,vehicle to pedestrian (V2P) communication). The optical communicationunit 440 is configured to perform communication with an external devicethrough the medium of light. Further, the optical communication unit 440may include a light emitting unit, which converts an electrical signalinto an optical signal and transmits the optical signal to the outside,and a light receiving unit which converts a received optical signal intoan electrical signal. The light emitting unit may be integrally formedwith a lamp provided included in the vehicle 100.

The broadcasting transmission and reception unit 450 is configured toreceive a broadcast signal from an external broadcasting managementserver or transmit a broadcast signal to the broadcasting managementserver through a broadcasting channel. The broadcasting channel mayinclude a satellite channel, and a terrestrial channel. The broadcastsignal may include a TV broadcast signal, a radio broadcast signal, anda data broadcast signal.

The ITS communication unit 460 may exchange information, data, orsignals with a traffic system. The ITS communication unit 460 canprovide acquired information or data to the traffic system. Further, theITS communication unit 460 may receive information, data, or signalsfrom the traffic system. For example, the ITS communication unit 460 mayreceive traffic information from the traffic system and provide thetraffic information to the controller 170. In another example, the ITScommunication unit 460 may receive a control signal from the trafficsystem, and provide the control signal to the controller 170 or aprocessor provided in the vehicle 100.

The processor 470 can control the overall operation of each unit of thecommunication device 400. The communication device 400 may include aplurality of processors 470, or may not include the processor 470. Whenthe communication device 400 does not include the processor 470, thecommunication device 400 can operate under the control of the controller170 or a processor of a device inside of the vehicle 100.

In addition, the communication device 400 may implement a vehicledisplay device, together with the user interface apparatus 200. In thisinstance, the vehicle display device may be referred to as a telematicsdevice or an Audio Video Navigation (AVN) device. The communicationdevice 400 can operate under the control of the controller 170.

The driving manipulation device 500 is configured to receive a userinput for driving the vehicle 100. In the manual mode, the vehicle 100can operate based on a signal provided by the driving manipulationdevice 500. The driving manipulation device 500 may include a steeringinput apparatus 510, an acceleration input device 530, and a brake inputdevice 570.

The steering input apparatus 510 receives a user input with regard tothe direction of travel of the vehicle 100. The steering input apparatus510 may take the form of a wheel to enable a steering input through therotation thereof and may be provided as a touchscreen, a touch pad, or abutton.

The acceleration input device 530 receives a user input for accelerationof the vehicle 100, and the brake input device 570 receives a user inputfor deceleration of the vehicle 100. Each of the acceleration inputdevice 530 and the brake input device 570 may take the form of a pedal.The acceleration input device or the brake input device may also beconfigured as a touch screen, a touch pad, or a button.

The driving manipulation device 500 can operate under the control of thecontroller 170. Further, the vehicle drive device 600 is configured toelectrically control the operation of various devices of the vehicle 100and may include a power train drive unit 610, a chassis drive unit 620,a door/window drive unit 630, a safety apparatus drive unit 640, a lampdrive unit 650, and an air conditioner drive unit 660.

The vehicle drive device 600 may further include other components inaddition to the aforementioned components, or may not include some ofthe aforementioned components. In addition, the vehicle drive device 600may include a processor. Each unit of the vehicle drive device 600 mayalso include its own processor.

The power train drive unit 610 controls the operation of a power trainand may include a power source drive unit 611 and a transmission driveunit 612. Further, the power source drive unit 611 can control a powersource of the vehicle 100.

When a fossil fuel-based engine is the power source, the power sourcedrive unit 611 can perform electronic control of the engine. As such thepower source drive unit 611 can control, for example, the output torqueof the engine. The power source drive unit 611 can also adjust theoutput toque of the engine under the control of the controller 170.

When an electric motor is the power source, the power source drive unit611 can control the motor. For example, the power source drive unit 610can control, for example, the RPM and toque of the motor under thecontrol of the controller 170.

The transmission drive unit 612 controls a transmission and adjusts thestate of the transmission. The transmission drive unit 612 can adjust astate of the transmission to a drive (D), reverse (R), neutral (N), orpark (P) state. Meanwhile, when an engine is the power source, thetransmission drive unit 612 can adjust a gear-engaged state to the driveposition D.

The chassis drive unit 620 can control the operation of a chassis andmay include a steering drive unit 621, a brake drive unit 622, and asuspension drive unit 623. The steering drive unit 621 can performelectronic control of a steering apparatus provided inside the vehicle100 and change the direction of travel of the vehicle 100.

The brake drive unit 622 can perform electronic control of a brakeapparatus provided inside the vehicle 100. For example, the brake driveunit 622 can reduce the speed of the vehicle 100 by controlling theoperation of a brake located at a wheel. In addition, the brake driveunit 622 can control a plurality of brakes individually and apply adifferent degree-braking force to each wheel.

The suspension drive unit 623 can perform electronic control of asuspension apparatus inside the vehicle 100. For example, when the roadsurface is uneven, the suspension drive unit 623 can control thesuspension apparatus so as to reduce the vibration of the vehicle 100.In addition, the suspension drive unit 623 can control a plurality ofsuspensions individually.

The door/window drive unit 630 can perform electronic control of a doorapparatus or a window apparatus inside the vehicle 100. In addition, thedoor/window drive unit 630 may include a door drive unit 631 and awindow drive unit 632.

The door drive unit 631 can control the door apparatus and controlopening or closing of a plurality of doors included in the vehicle 100.Further, the door drive unit 631 can control opening or closing of atrunk or a tail gate and control opening or closing of a sunroof.

Further, the window drive unit 632 can perform electronic control of thewindow apparatus and control opening or closing of a plurality ofwindows included in the vehicle 100. The safety apparatus drive unit 640can perform electronic control of various safety apparatuses providedinside the vehicle 100 and may include an airbag drive unit 641, asafety belt drive unit 642, and a pedestrian protection equipment driveunit 643.

The airbag drive unit 641 can perform electronic control of an airbagapparatus inside the vehicle 100. For example, upon detection of adangerous situation, the airbag drive unit 641 can control an airbag tobe deployed. In addition, the safety belt drive unit 642 can performelectronic control of a seatbelt apparatus inside the vehicle 100. Forexample, upon detection of a dangerous situation, the safety belt driveunit 642 can control passengers to be fixed onto seats 110FL, 110FR,110RL, and 110RR with safety belts.

The pedestrian protection equipment drive unit 643 can performelectronic control of a hood lift and a pedestrian airbag. For example,upon detection of a collision with a pedestrian, the pedestrianprotection equipment drive unit 643 can control a hood lift and apedestrian airbag to be deployed.

In addition, the lamp drive unit 650 can perform electronic control ofvarious lamp apparatuses provided inside the vehicle 100, and the airconditioner drive unit 660 can perform electronic control of an airconditioner inside the vehicle 100. For example, when the innertemperature of the vehicle 100 is high, the air conditioner drive unit660 can operate the air conditioner so as to supply cool air to theinside of the vehicle 100.

In addition, the vehicle drive device 600 may include a processor. Eachunit of the vehicle dive device 600 may also include its own processor.Further, the vehicle drive device 600 can operate under the control ofthe controller 170. The operation system 700 is a system for controllingthe overall driving operation of the vehicle 100 and can operate in theautonomous driving mode.

The operation system 700 may include the driving system 710, the parkingout system 740, and the parking system 750. The operation system 700 mayfurther include other components in addition to the aforementionedcomponents, or may not include some of the aforementioned component.

In addition, the operation system 700 may include a processor. Each unitof the operation system 700 may also include its own processor. When theoperation system 700 is implemented as software, the operation system700 may be a subordinate concept of the controller 170. The operationsystem 700 may be a concept including at least one of the user interfaceapparatus 200, the object detection device 300, the communication device400, the driving manipulation device 500, the vehicle drive device 600,the navigation system 770, and the sensing unit 120, and the controller170.

The driving system 710 can perform driving of the vehicle 100 byproviding a control signal to the vehicle drive device 600 in responseto reception of navigation information from the navigation system 770,in response to reception of object information from the object detectiondevice 300, and in response to reception of a signal from an externaldevice through the communication device 400.

Further, the driving system 710 may include at least one of the userinterface apparatus 200, the object detection device 300, thecommunication device 400, the driving manipulation device 500, thevehicle drive device 600, the navigation system 770, the sensing unit120, and the controller to perform driving of the vehicle 100. Thedriving system 170 can thus be referred to as a vehicle driving controlapparatus.

Further, the vehicle pulling-out system 740 can park the vehicle 100 outof a parking space. For example, the vehicle pulling-out system 740 canmove the vehicle 100 out of a parking space, by providing a controlsignal to the vehicle drive device 600 in response to reception ofnavigation information from the navigation system 770, in response toreception of object information from the object detection device 300,and in response to reception of a signal from an external device.

In addition, the vehicle pulling-out system 740 may include at least oneof the user interface apparatus 200, the object detection device 300,the communication device 400, the driving manipulation device 500, thevehicle drive device 600, the navigation system 770, the sensing unit120, and the controller 170 to park the vehicle 100 out of a parkingspace. The vehicle pulling-out system 740 may be referred to as avehicle pulling-out control apparatus.

In addition, the parking system 750 can park the vehicle 100 in aparking space, by providing a control signal to the vehicle drive device600 in response to reception of navigation information from thenavigation system 770, in response to reception of object informationfrom the object detection device 300, and in response to reception of asignal from an external device. The parking system 750 may include atleast one of the user interface apparatus 200, the object detectiondevice 300, the communication device 400, the driving manipulationdevice 500, the vehicle drive device 600, the navigation system 770, thesensing unit 120, and the controller 170 to park the vehicle 100 in aparking space. The parking system 750 can thus be referred to as avehicle parking control apparatus.

In addition, the navigation system 770 can provide navigationinformation including at least one of map information, information on aset destination, information on a route to the set destination,information on various objects along the route, lane information, andinformation on the current location of a vehicle. The navigation system770 may also include a memory and a processor. The memory can storenavigation information, and the processor can control the operation ofthe navigation system 770. The navigation system 770 can also updatepre-stored information by receiving information from an external devicethrough the communication device 400. The navigation system 770 may beclassified as an element of the user interface apparatus 200.

In addition, the sensing unit 120 can sense the state of the vehicle.The sensing unit 120 may include an attitude sensor (for example, a yawsensor, a roll sensor, or a pitch sensor), a collision sensor, a wheelsensor, a speed sensor, a gradient sensor, a weight sensor, a headingsensor, a gyro sensor, a position module, a vehicle forward/reversemovement sensor, a battery sensor, a fuel sensor, a tire sensor, asteering sensor based on the rotation of the steering wheel, anin-vehicle temperature sensor, an in-vehicle humidity sensor, anultrasonic sensor, an illumination sensor, an accelerator pedal positionsensor, and a brake pedal position sensor.

The sensing unit 120 can also acquire sensing signals with regard to,for example, vehicle attitude information, vehicle collisioninformation, vehicle driving direction information, vehicle locationinformation (GPS information), vehicle angle information, vehicle speedinformation, vehicle acceleration information, vehicle tilt information,vehicle forward/reverse movement information, battery information, fuelinformation, tire information, vehicle lamp information, in-vehicletemperature information, in-vehicle humidity information, steering-wheelrotation angle information, out-of-vehicle illumination information,information about the pressure applied to an accelerator pedal, andinformation about the pressure applied to a brake pedal.

The sensing unit 120 may further include, for example, an acceleratorpedal sensor, a pressure sensor, an engine speed sensor, an AirFlow-rate Sensor (AFS), an Air Temperature Sensor (ATS), a WaterTemperature Sensor (WTS), a Throttle Position Sensor (TPS), a Top DeadCenter (TDC) sensor, and a Crank Angle Sensor (CAS). The sensing unit120 may generate vehicle state information based on sensing data.Further, the vehicle state information may be information that isgenerated based on data sensed by a variety of sensors inside a vehicle.

For example, the vehicle state information may include vehicle attitudeinformation, vehicle speed information, vehicle tilt information,vehicle weight information, vehicle direction information, vehiclebattery information, vehicle fuel information, vehicle tire pressureinformation, vehicle steering information, in-vehicle temperatureinformation, in-vehicle humidity information, pedal positioninformation, vehicle engine temperature information, etc.

The interface 130 serves as a passage for various kinds of externaldevices that are connected to the vehicle 100. For example, theinterface 130 may have a port that is connectable to a mobile terminaland may be connected to the mobile terminal via the port. In thisinstance, the interface 130 can exchange data with the mobile terminal.

In addition, the interface 130 can serve as a passage for the supply ofelectrical energy to a mobile terminal connected thereto. When themobile terminal is electrically connected to the interface 130, theinterface 130 can provide electrical energy, supplied from the powersupply unit 190, to the mobile terminal under the control of thecontroller 170.

Further, the memory 140 is electrically connected to the controller 170and can store basic data for each unit, control data for the operationalcontrol of each unit, and input/output data. The memory 140 may be anyof various hardware storage devices, such as a ROM, a RAM, an EPROM, aflash drive, and a hard drive. Further, the memory 140 can store variousdata for the overall operation of the vehicle 100, such as programs forthe processing or control of the controller 170.

In addition, the memory 140 may be integrally formed with the controller170, or may be provided as an element of the controller 170. Thecontroller 170 controls the overall operation of each unit inside thevehicle 100 and may be referred to as an Electronic Control Unit (ECU).Further, the power supply unit 190 can supply power required to operateeach component under the control of the controller 170. In particular,the power supply unit 190 can receive power from, for example, a batteryinside the vehicle 100.

At least one processor and the controller 170 included in the vehicle100 may be implemented using at least one of Application SpecificIntegrated Circuits (ASICs), Digital Signal Processors (DSPs), DigitalSignal Processing Devices (DSPDs), Programmable Logic Devices (PLDs),Field Programmable Gate Arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, and electric units for theimplementation of other functions.

Next, FIG. 8 is a block diagram illustrating a user interface apparatus200 for a vehicle according to an embodiment of the present invention.Referring to FIG. 8, the user interface apparatus 200 for a vehicle mayinclude an input unit, an internal camera 220, a biometric sensing unit230, a memory 240, an interface 245, an output unit 250, a processor270, and a power supply unit 290.

The user interface apparatus 200 may further include other components tothe aforementioned components, or may not include some of theaforementioned components. The user interface apparatus 200 shown inFIG. 8 includes each elements of the user interface apparatus 200described with reference to FIG. 7. Hereinafter, a redundant descriptionof a redundant feature is omitted, and a feature not described withreference to FIG. 7 is described primarily.

The user interface apparatus 200 according to an embodiment of thepresent invention may be referred to a vehicle display apparatus, aCenter Information Display (CID), or an Audio Video Navigation (AVN)system. The user interface apparatus 200 and the communication device400 may be combined to implement a telematics device.

The input unit 210, the internal camera 220, and the biometric sensingunit 230 may be the same as described with reference to FIG. 7. Further,the input unit 210 can detect a gesture of a user. The internal camera220 can detect a gaze of a driver. The memory 240 is electricallyconnected to the processor 270. Further, the memory 240 may store basicdata for each unit, control data for the operational control of eachunit, and input/output data. The memory 240 may be any of varioushardware storage devices, such as a ROM, a RAM, an EPROM, a flash drive,and a hard drive. The memory 240 may store various data for the overalloperation of the user interface apparatus 200, such as programs for theprocessing or control of the processor 270.

The memory 240 may be integrally formed with the processor 270, or maybe an element of the processor 270. Further, the interface 245 canexchange information, data, or a signal with a different device includedin the vehicle 100. In addition, the interface 245 can transmit receivedinformation, data, or signal to the processor 270. The interface 245 canalso transmit information, data, or signal generated or processed by theprocessor 270 to a different device included in the vehicle 100. Theinterface 245 can receive information, data, or signal from a differentdevice included in the vehicle 100.

The interface 245 can receive a view of an area outside the vehicle 100from the object detection device 300. The interface 245 can also receiveobject information from the object detection device 300. For example,the interface 245 can receive information on disparity of multipleobjects included in a view of an area outside the vehicle 100.

The interface 245 can receive emergency situation information from theobject detection device 300. In addition, the object detection device300 can generate emergency situation information based on information onTime to Collision (TTC) between the vehicle 100 and an object outsidethe vehicle 100. Further, the emergency situation information can begenerated based on the information on TTC between the vehicle 100 andthe object outside the vehicle 100.

The object detection device 300 can generate emergency situationinformation based on information on a distance between the vehicle 100and an object outside the vehicle 100, and information on a speed of thevehicle 100 relative to the object outside the vehicle 100. Theemergency situation information may be generated based on theinformation on the distance between the vehicle 100 and an objectoutside the vehicle 100, and the information on the speed of the vehicle100 relative to the object outside the vehicle 100. For example, theemergency situation information may be information on a predictedcollision between the vehicle 100 and an object.

The interface 245 can receive navigation information from the navigationsystem 770. The information, data, or signal received by the interface245 may be provided to the processor 270. Further, the output unit 250may include a display unit, a sound output unit 252, and a haptic outputunit 253.

The output unit 250 may be the same as described with reference to FIG.7. Hereinafter, the display unit 251 is described primarily. The displayunit 251 can operate under control of the processor 270. Further, thedisplay unit 251 may include a touch screen. The processor 270 cancontrol overall operation of each unit of the user interface apparatus200.

The processor 270 can receive information on detection of a usergesture. The processor 270 can also determine, based on a motiontrajectory of a gesture, a direction from which the gesture is applied.For example, the processor 170 can determine, based on a start point ofa gesture motion, whether the gesture is applied from the driver seat ora front passenger seat.

The processor 270 can also control the display unit 251 to display apreset different screen based on whether a gesture is applied from adriver seat or a front passenger seat. If it is determined that agesture detected by the gesture input unit 212 is a gesture applied fromthe driver seat, the processor 270 can control the display unit 251 todisplay a preset first screen or a preset second screen.

If it is determined that a gesture detected by the gesture input unit212 is a gesture applied from the front passenger seat, the processor270 can control the gesture input unit 212 to display a preset thirdscreen. The third screen may include a screen for operation of a frontpassenger seat convenient device. For example, the third screen mayinclude at least one of the following: a screen for operation of a frontpassenger seat position control device, a screen for operation of afront passenger seat window control device, and a screen for operationof a front passenger seat door control device. Alternatively, the thirdscreen may include a screen for operation of a vehicle environmentcontrol apparatus. For example, the third screen may include a screenfor operation of an air conditioner or a screen for operation of a roomlamp.

In addition, the processor 270 can divide the display unit 251 intomultiple regions and can control the display unit 251 to display apreset different screen based on which region to which a gesture isapplied among the multiple regions. If it is determined that a gestureis applied from the driver seat and that the gesture applied from thedriver seat is a gesture applied to a first region among the multipleregions, the processor 270 can control the display unit 251 to display afirst screen.

The first screen may include a screen for operation of the vehicle drivedevice 600. For example, the first screen may include a screen foroperation which is designed to control at east one of the power traindrive unit 610, the chassis drive unit 620, the door/window drive unit630, the safety apparatus drive unit 640, the lamp drive unit 650, andthe air conditioner drive unit 660.

If it is determined that a gesture is applied from the driver seat andthat the gesture applied from the driver seat is a gesture applied to asecond region among the multiple regions, the processor 270 can controlthe display unit 251 to display a second screen. The second screen mayinclude a screen for operation of a vehicle utility device. For example,the second screen may include at least one of the following: a screenfor operation of a navigation device, a screen for operation of acommunication device, a screen for operation of a music play device, ascreen for operation of a schedule management device, and a screen foroperation of a video play device.

In addition, the processor 270 can receive information on a speed of thevehicle 100 via the interface 245 and can control the display unit 251based on the information on the speed of the vehicle 100 to disabledisplaying of a screen. For example, if the speed of the vehicle 100 isa preset speed or higher, the processor 270 can display the display unit251, so that nothing is displayed on the screen.

Also, if the speed of the vehicle 100 is a preset speed or higher, theprocessor 270 can control the display unit 251 to display only a screenrelated to a preset application. For example, if the speed of thevehicle 100 is a preset speed or higher when the display unit 251includes a touch screen, the processor 270 can control the display unit251 to disable displaying of a screen and applying of a touch input viathe touch screen. As such, the display unit 251 is controlled based oninformation on a speed of the vehicle 100, and therefore, a driver, whois driving the vehicle 100 at high speeds, can focus only on thedriving, thereby preventing an accident.

The processor 270 can collect traffic flow information via the interface245. In addition, traffic flow may be a value that is calculated basedon the number of vehicles located within a preset range. Further, thetraffic flow information may be generated by an external server.

In addition, the processor 270 can receive traffic flow information fromthe communication device 400 via the interface 245. The processor 270can then control the display unit 251 based on traffic flow informationto disable displaying of a screen. For example, if the traffic flow isequal to or greater than a reference value, the processor 270 cancontrol the display unit 251, so that nothing is displayed on thescreen, so that only a screen related to a preset application isdisplayed, etc. For example, if the traffic flow is equal to or greaterthan a reference value when the display unit 251 includes a touchscreen, the processor 270 can control the display unit 251 to disabledisplaying of a screen and applying of a touch input via the touchscreen.

In addition, the processor 270 can receive information on a degree ofdriving difficulty in a travel roadway via the interface 245. The degreeof driving difficulty may be a value calculated based on at least one ofthe following: whether there is a curve, a curvature value of the curve,whether there is a slope, a gradient of the slope, whether there is anintersection, whether there is an entrance ramp, whether there is anexit ramp, whether there is a tunnel, etc. Further, the information onthe degree of driving difficulty may be generated by an external server.The processor 270 can receive the information on the degree of drivingdifficulty from the communication device 400 via the interface 245.

Further, the processor 270 can control the display unit 251 based on theinformation on the degree of driving difficulty to disable displaying ofa screen. For example, if a degree of driving difficulty is equal to orgreater than a reference value, the processor 270 can control thedisplay unit 251, so that nothing is displayed on the screen, so thatonly a screen related to a preset application is displayed, etc. If adegree of driving difficulty is equal to or greater than a referencevalue when the display unit 251 includes a touch screen, the processor270 can control the display unit 251 to disable displaying of a screenand applying of a touch input via the touch screen. The processor 270can control the display unit 251 based on driver gaze information todisable displaying of a screen.

In addition, the processor 270 can detect a gaze of a driver via theinternal camera 220, and generate information on the gaze of the driver.For example, the processor 270 can detect a pupil of a driver, andgenerate information on a gaze of the driver by tracking movement of thepupil. The display unit 251 may also include a touch screen. If it isdetermined that a gaze of a driver has been directed toward the displayunit 251 for a preset period of time or more, the processor 270 cancontrol the display unit 251 to disable displaying of a screen on thetouch screen or applying a touch input to the touch screen.

As discussed above, the vehicle 100 may be an autonomous vehicle. Inmore detail, the vehicle 100 may be driven in an autonomous drivingstate or a manual driving state. If it is determined that a gesture isapplied from a driver seat while the vehicle 100 is in the manualdriving state, the processor 270 can determine whether a criteria forswitching to an autonomous driving state is satisfied. If it isdetermined that the criteria for switching to the autonomous drivingstate is satisfied, the processor 270 can provide a signal to switch thestate of the vehicle 100 to the autonomous driving state.

The criteria for switching to an autonomous driving state may be basedon at least one of the following: information on execution of a presetapplication, information about entrance in a preset level or higher of apreset application, driver gaze information, information on a degree ofdriving difficulty, speed information, and traffic flow information. Forexample, if a preset application is executed while the vehicle 100 is inthe manual driving state, the processor 270 can provide a signal so thatthe state of the vehicle 100 is switched to the autonomous drivingstate. When entering a preset depth level or higher of a presetapplication in response to a user input when the preset application isbeing executed, the processor 270 can provide a signal so that a stateof the vehicle 100 is switched from a manual driving state to anautonomous driving state.

A depth level may be defined as the number of user inputs that arerequired for execution of a function associated with a correspondingitem. Further, the higher depth an item has, the more user inputs a userneeds to apply to execute a desired function. Accordingly, if a driverbehind the wheel wants to execute a function of an item having a higherdepth, the driver may not look ahead down the road for a longer time.Thus, when entering a preset depth level or higher of the application,the state of the vehicle 100 can be switched to an autonomous drivingstate, so that a possible accident can be prevented.

For example, if a gaze of a driver has been directed toward the displayunit 251 for a predetermined time or more, the processor 270 can providea signal to switch the state of the vehicle 100 from the manual drivingstate to the autonomous driving state. In addition, if a degree ofdriving difficulty of a roadway being travelled by the vehicle 100 isequal to or greater than a reference value, the processor 270 canprovide a signal to switch the state of the vehicle 100 from the manualdriving state to the autonomous driving state.

For example, if a speed of the vehicle 100 is equal to or greater than areference value, the processor 270 can provide a signal to switch thestate of the vehicle 100 from the manual driving state to the autonomousdriving state, and if traffic flow is equal to or greater than areference value, the processor 270 can provide a signal to switch thestate of the vehicle 100 from the autonomous driving state to the manualdriving state. If a criteria for switching to an autonomous drivingstate is satisfied, the processor 270 can control the display unit 251to display a screen corresponding to autonomous driving. The screencorresponding to autonomous driving may include a brake input button ofthe vehicle 100 or a steering input button of the vehicle 100.

The screen corresponding to autonomous driving may include a screen foroperation of a vehicle utility device. For example, the screencorresponding to autonomous driving may include at least one of thefollowing: a screen for operation of a navigation device, a screen foroperation of a communication device, a screen for operation of a musicplay device, a screen for operation of a schedule management device, anda screen for operation of a video play device.

The screen corresponding to autonomous driving may include a contentprovision screen. For example, the screen corresponding to autonomousdriving may include at least one of a screen of providing image contentsand a screen of providing music contents. If emergency situationinformation is received on the interface 245, the processor 270 cancontrol the display 251 to display a brake input button or a steeringinput button.

If a user input is received via the brake input button or the steeringinput bottom after a gesture detected by the gesture input unit 212 isdetermined to be a gesture applied from a front passenger seat, theprocessor 270 can provide a control signal to operate a brake apparatusor a steering apparatus. In such an emergency situation, it is possibleto avert an emergency situation in response not only to a user inputapplied from the driver seat, but from to a user input applied from thefront passenger seat, so that an accident can be prevented. Undercontrol of the processor 270, the power supply unit 290 can providepower required for operation of each component. In particular, the powersupply unit 290 may be provided with power from a battery inside thevehicle 100.

Next, FIG. 9 is a flowchart illustrating how a user interface apparatusfor a vehicle operates according to an embodiment of the presentinvention. Referring to FIG. 9, the processor 270 can receiveinformation on detection of a gesture of a user in S910. The processor270 can then determine, based on a trajectory of movement of thegesture, a direction from which the gesture is applied in S920.Specifically, the processor 270 can determine whether the gesture isapplied from the driver seat or from the front passenger seat.

As discussed above, the processor 270 can divide the display unit 251into multiple regions. Specifically, the processor 270 can divide thedisplay unit 251 into a first region and a second region. When it isdetermined that the gesture is be applied from the driver seat, theprocessor 270 determines which region to which the gesture is appliedamong the multiple regions in S930. Specifically, the processor 270determines whether the gesture is applied to the first region or thesecond region.

When it is determined that the gesture is applied to the first region inS930, the processor 270 displays a first screen in S940. The firstscreen may include a screen for operation of the vehicle drive device600. For example, the first screen may include a screen for operation ofthe power train drive unit 610, the chassis drive unit 620, thedoor/window drive unit 630, the safety apparatus drive unit 640, thelamp drive unit 650, and the air conditioner drive unit 660.

Then, the processor 270 determines whether a criteria for switching toan autonomous driving state is satisfied in S950. The criteria forswitching to the autonomous driving state may be based on at least oneof the following: information on execution of a preset application,information about entrance in a preset level or higher of a presetapplication, driver gaze information, information on a degree of drivingdifficulty, speed information, and traffic flow information.

If it is determined that the criteria for switching to the autonomousdriving state is satisfied in S950, the processor 270 can provide asignal to switch the state of the vehicle 100 from the manual drivingstate to the autonomous driving state in S960. When the state of thevehicle 100 is switched to the autonomous driving state, the processor270 can control the display unit 251 to display a screen correspondingto autonomous driving in S970.

Meanwhile, if it is determined in operation S930 that the gesture isapplied, not to the first region, but to the second region, theprocessor 270 displays a second screen in S980. The second screen mayinclude a screen for operation of a vehicle utility device. For example,the second screen may include at least one of the following: a screenfor operation of a navigation device, a screen for operation of acommunication device, a screen for operation of a music play device, ascreen for operation of a schedule management device, and a screen foroperation of a video play device.

Meanwhile, if it is determined in S920 that a gesture is applied, notfrom the driver seat, but from the front passenger seat, the processor270 can display a third screen in S990. The third screen may include ascreen for operation of a front passenger seat utility device. Forexample, the third screen may include at least one of the following: ascreen for operation of a front passenger seat position control device,a screen for operation of a front passenger seat window control device,and a screen for operation of a front passenger seat door controldevice. Alternatively, the third screen may include a screen foroperation of a vehicle environment control apparatus. For example, thethird screen may include a screen for operation of an air conditioner ora screen for operation of a room lamp.

Next, FIGS. 10 to 12 illustrate examples of how a user interfaceapparatus for a vehicle operates in response to a gesture inputaccording to an embodiment of the present invention. As shown in 1010 ofFIG. 10, the gesture input unit 212 can detect gestures 1011 and 1012.The gestures 1011 and 1012 may include a hovering gesture and a touchgesture. Further, the hovering gesture may be a gesture input based on atrajectory of a user's finger that moves at a predetermined distance ormore from the display unit 251.

Based on the trajectory of movement of each of the gestures 1011 and1012, the processor 270 can determine a direction from which each of thegestures is applied. For example, the processor 270 can determine thatthe gesture 1011 moving from the driver seat toward the display unit 251is a gesture applied from the driver seat, or the processor 270 candetermine that the gesture 1012 moving from the front passenger seattoward the display unit 251 is a gesture applied from the frontpassenger seat.

On the basis that the gesture 1011 or 1012 is applied from the driverseat or the front passenger seat, the processor 270 can control thedisplay unit 251 to display a preset different screen. If it isdetermined that the gesture 1011 is a gesture applied from the driverseat, the processor 270 can control the display unit 251 to display apreset first screen or a preset second screen, as shown in 1040 of FIG.10.

The first screen may include a screen for operation of the vehicle drivedevice 600. For example, the first screen may be a screen for operationthat is designed to control at least one of the power train drive unit610, the chassis drive unit 620, the door/window drive unit 630, thesafety apparatus drive unit 640, the lamp drive unit 650, and the airconditioner drive unit 660.

The second screen may include a screen for operation of a vehicleutility device. For example, the second screen may include at least oneof the following: a screen for operation of a navigation device, ascreen for operation of a communication device, a screen for operationof a music play device, a screen for operation of a schedule managementapparatus, and an image play device. If it is determined that thegesture 1012 is a gesture applied from the front passenger seat, theprocessor 270 can control the display unit 251 to display a preset thirdscreen, as shown in 1070 of FIG. 10.

The third screen may include a screen for operation of a front passengerseat utility device. For example, the third screen may include at leastone of the following: a screen for operation of a front passenger seatposition control device, a screen for operation of a front passengerseat window control device, and a screen for operation of a frontpassenger seat door control device. Alternatively, the third screen mayinclude a screen for operation of a vehicle environment controlapparatus, and the third screen may include a screen for operation of anair conditioner or a screen for operation of a room lamp.

As shown in FIG. 11, the processor 270 can divide the display unit 251into a plurality of regions. For example, the processor 270 can dividethe display unit 251 into a first region 1111 and a second region 1112.The processor 270 can control the display unit 251 to display a presetdifferent screen based on which region to which a gesture is appliedamong the plurality of regions.

As shown in 1110 and 1130 of FIG. 11, if it is determined that a gesture1113 is a gesture applied to the first region 1111, the processor 270can control the display unit 251 to display a first screen 1131. Inaddition, the processor 270 can control the first screen 1131 to bedisplayed in the first region 1111, the second region 1112, or theentire region of the display unit 251.

The first screen 1131 may include a screen for operation of the vehicledrive device 600. For example, the first screen may include a screen foroperation to control at least one of the power train drive unit 610, thechassis drive unit 620, the door/window drive unit 630, the safetyapparatus drive unit 640, the lamp drive unit 650, and the airconditioner drive unit 660.

As shown in 1160 and 1180 of FIG. 11, if it is determined that thegesture 1163 is a gesture applied to the second region 1112, theprocessor 270 can control the display unit 251 to display the secondscreen 1181. In addition, the processor 270 can control the secondscreen 1181 to be displayed in the first region 1111, the second region1112, or the entire region of the display unit 251.

The second screen may include a screen for operation of a vehicleutility device. For example, the second screen may include at least oneof the following: a screen for operation of a navigation device, ascreen for operation of a communication device, a screen for operationof a music play device, a screen for operation of a schedule managementdevice, and a screen for operation of a video play device.

As illustrated in FIG. 12, the processor 270 can divide the display unit251 into a plurality of regions. For example, the processor 270 candivide the display unit 251 into a first region 1111 and a second region1112. The processor 270 can control the display unit 251 to display apreset different screen based on whether a gesture applied to the firstregion among the plurality of regions is a gesture applied from thedriver seat or from the front passenger seat.

If it is determined that a gesture 1211 applied to the first region 1111is a gesture applied from the driver seat, the processor 270 can controlthe display unit 251 to display a preset fourth screen 1241, as shown in1241 of FIG. 12. In addition, the processor 270 can control the fourthscreen 1241 to be displayed in the first region 1111, the second region1112, or the entire region of the display unit 251.

The fourth screen may include a screen for operation of a driver seatutility device. For instance, the fourth screen may include at least oneof the following: a screen for operation of a driver seat positioncontrol device, a screen for operation of a driver seat window controldevice, and a screen for operation of a driver seat door control device.

In another example, the fourth screen may include a screen for operationof an all seat utility device. For instance, the fourth screen mayinclude at least one of the following: a screen for operation of an allseat position control device, a screen for operation of an all seatwindow control device, and a screen for operation of an all-seat doorcontrol device.

In yet another example, the fourth screen may include a screen foroperation of any of various apparatuses included in a vehicle. Forinstance, the fourth screen may include at least one of the following: ascreen for operation of an air conditioner, a screen for operation of aroom lamp, a screen for operation of a head lamp, a screen for operationof wipers, and a screen for operation of a turn-signal lamp.

If it is determined that a gesture 1212 applied to the first region 1111is a gesture applied from the front passenger seat, the processor 270can control the display unit 251 to display a preset fifth screen 1271,as shown in 1270 of FIG. 12. In addition, the processor 270 can controlthe fifth screen 1271 to be displayed in the first region 1111, thesecond region 1112, or the entire region of the display unit 251.

The fifth screen 1271 may include a screen for operation of a frontpassenger seat utility device. For instance, the fifth screen 1271 mayinclude at least one of a screen for operation of a front passenger seatposition control device, a screen for operation of a front passengerseat window control device, and a screen for operation of a frontpassenger seat door control device. In another example, the fifth screen1271 may include a screen for operation of a vehicle environment controlapparatus. For instance, the fifth screen 1271 may include a screen foroperation of an air conditioner or a screen for operation of a roomlamp.

Next, FIG. 13 is a flowchart illustrating how a user interface apparatusfor a vehicle operates according to an embodiment of the presentinvention. Referring to FIG. 13, the processor 270 can receiveinformation on a speed of the vehicle 100 from the sensing unit 120 orthe navigation system 770 via the interface 245 in S1310.

The processor 270 can then control the display unit 251 based oninformation on a speed of the vehicle 100 to disable displaying of ascreen in S1320. For example, if a speed of the vehicle 100 is equal toor greater than a preset value while information on detection of a usergesture is received in S910, the processor 270 can disable displaying ofa screen according to a gesture, and if a speed of the vehicle 100 isequal to or greater than a preset value, the processor 270 can controlthe display unit 251, so that only a screen related to a presetapplication is displayed.

For example, if a speed of the vehicle 100 is equal to or greater than apreset value when the display 251 includes a touch screen, the processor270 can control the display 251 to disable displaying of a screen andapplying of a touch input via the touch screen. If a preset user inputis received while displaying of a screen is disabled, the processor 270can control an icon of a preset application to be displayed in S1330.Here, the preset user input may include a gesture input, a voice input,and a touch input. If a user input is received to select a displayedicon, the processor 270 can control an application corresponding to theicon to be executed in S1340.

Next, FIGS. 14 to 17 are examples in which a user interface apparatusexecutes an application when displaying a screen is disabled accordingto an embodiment of the present invention. The processor 270 can acquireinformation on a speed of the vehicle 100 and control the display unit251 based on the information on the speed of the vehicle 100.

As shown in 1410 of FIG. 14, when the vehicle 100 is stopped or travelsat a speed equal to or smaller than a preset value, the processor 270can control the display unit 251 to display a specific screen. Forexample, the processor 270 can control the display unit 251 to display,in the second region 1112, a screen that includes a plurality of iconsrespectively corresponding to a plurality of applications, the currenttime, the current day of week, the current date, etc. The processor 270can also control the display unit 251 to display, in the first region1111, a screen that includes vehicle state information. The vehiclestate information may include indoor temperature information, outdoortemperature information, and air-conditioning state information.

As shown in 1430 of FIG. 14, if a speed of the vehicle 100 is equal toor greater than a preset value, the processor 270 can disable displayingof a screen. In this instance, the processor 270 can display information1431 indicating that displaying of a screen is disabled. In addition,the processor 270 can control the display unit 251 not to display anapplication screen and icons corresponding to applications. Even whendisplaying of a screen is disabled, the processor 270 can control thedisplay unit 251 to display information on weather, the current time,the current day of week, the current date, etc. Even when displaying ofa screen is disabled, the processor 270 can control the display unit 251to display a screen including vehicle state information in the firstregion 1111.

As shown in 1460 of FIG. 14, if a preset user input is received whiledisplaying of a screen is disabled, the processor 270 cancan perform acontrol action to display an icon 1461 of a preset application. Thepreset application may be an application that helps a driver to drive avehicle. For example, the preset application may include anavigation-related application, an Advanced Driver Assistance System(ADAS)-related application, a driving situation information-relatedapplication, and an autonomous driving-related application.

In 1460 of FIG. 14, a multi-touch input is taken as an example of a userinput, but the user input may be a hovering gesture input or a voiceinput. As shown in 1480 of FIG. 14, when a user input is received toselect an icon, the processor 270 can execute an applicationcorresponding to the icon. In 1460 of FIG. 14, a touch input is taken asan example of a user input, but the user input may be a hovering gestureinput or a voice input.

As such, a vehicle driving-related application may be driven even whendisplaying of a screen is disabled, and therefore, safe driving may bemaintained and user convenience may improve. As shown in 1510 of FIG.15, a first application may be displayed on the display unit 251 whendisplaying of a screen is disabled. In 1510 of FIG. 15, anavigation-related application is taken as an example of a user input.

While the first application is displayed, the processor 270 can receivea user input 1512 to change a displayed application. In 1510 of FIG. 15,a flicking touch input is taken as an example of a user input, but theuser input may be a hovering gesture input or a voice input. When theuser input 1512 is received, the processor 270 can control the displayunit 251 to display a second application, as shown in 1530 of FIG. 15.In 1530 of FIG. 15, an Around View Monitoring (AVM) application, whichis one of ADAS-related applications, is taken as an example of thesecond application. In this instance, the processor 270 can execute thefirst application, which was previously executed and displayed, inbackground.

While the second application is displayed, the processor 270 can receivea user input 1532 for changing a displayed application. In 1530 of FIG.15, a flicking touch input is taken as an example of a user input, butthe user input may be a hovering gesture input or a voice input. Whenthe user input 1532 is received, the processor 270 can control thedisplay unit 251 to display a third application.

In 1560 of FIG. 15, an autonomous driving-related application is takenas an example of the third application. In this instance, the processor270 can execute the first or second application, which was previouslyexecuted or displayed, in background. In addition, the order ofdisplaying the first to third applications according to a user input maybe set in advance. The order may be changed by a user input.

As shown in 1610 of FIG. 16, the processor 270 can receive a touch inputon a setting icon 1611. In this instance, as shown in 1630 of FIG. 16,the processor 270 can display a setup screen. Further, the setup screenmay be a screen for selecting an application that is executable whendisplaying of a screen is disabled. The processor 270 can display, onthe setup screen, a plurality of icons respectively corresponding to aplurality of applications

While the setup screen is displayed, the processor 270 can receive atouch input on a first icon 1601 corresponding to a first application.Then, as shown in 1660 of FIG. 16, the processor 270 can receive a touchinput on a second icon 1602 corresponding to a second application. Then,as shown in 1680 of FIG. 16, the processor 270 can receive a touch inputon a third icon 1603 corresponding to a third application. Then, theprocessor 270 can receive a touch input on a done button 1604. Thus, anyone of the first to third applications may be selected as an applicationthat is executable when displaying of a screen is disabled. In thisinstance, the order of displaying applications may correspond to theorder of selection of the applications.

As shown in 1710 of FIG. 17, if a speed of the vehicle 100 is equal toor greater than a preset value, the processor 270 can disable displayingof a screen. In this instance, the processor 270 can display information1431 indicating that displaying of a screen is disabled, and control thedisplay unit 251 not to display an application screen and iconscorresponding to applications.

Even when displaying of a screen is disabled, the processor 270 cancontrol the display unit 251 to display information on weather, thecurrent time, the current day of week, and the current date. Even whendisplaying of a screen is disabled, the processor 270 can control thedisplay unit 251 to display call receipt information and message receiptinformation 1711.

As shown in 1730 and 1760 of FIG. 17, if a first user input is receivedwhen displaying of a screen is disabled, the processor 270 can perform acontrol action to display a first application screen. In addition, if asecond user input is received when displaying of a screen is disabled,the processor 270 can perform a control action to display a secondapplication screen.

In addition, if a third user input is received when displaying of ascreen is disabled, the processor 270 can perform a control action todisplay a third application screen. Matching of the first to third userinputs with the first and third applications may be readily set.Alternatively, The processor 270 can match the first to third userinputs with the first to third applications based on user inputs.

FIG. 18 is a diagram illustrating how an application corresponding to astate of a vehicle is executed, according to an embodiment of thepresent invention. As shown in 1810 of FIG. 18, when the vehicle 100 isin a first state, the processor 270 can disable displaying of a screen.In this instance, the processor 270 can control the display unit 251 todisplay first state information 1811 of the vehicle 100.

As shown in 1830 of FIG. 18, when displaying of a screen is disabled,the processor 270 can receive a preset user input. In this instance, theprocessor 270 can perform a control action to display icons 1831 and1832 of a preset application corresponding to the first state of thevehicle 100. For example, an application corresponding to a parked statemay include an AVM application and a parking assist application. If auser's input on any one of the display icons 1831 and 1832 is received,the processor 270 can control the display unit 251 to display anapplication screen corresponding to autonomous driving.

As shown in 1860 of FIG. 18, when the vehicle 100 is in a second state,the processor 270 can disable displaying of a screen. In this instance,the processor 270 can control the display unit 251 to display secondstate information 1861 of the vehicle 100. In 1860 of FIG. 18, a drivingstate is taken as an example of the second state of the vehicle 100.

As shown in 1880 of FIG. 18, when displaying of a screen is disabled,the processor 270 can receive a preset user input. In this instance, theprocessor 270 can perform a control action to display icons 1881 and1882 of a preset application corresponding to the second state. Forexample, an application corresponding to a driving state may include anavigation application, an Autonomous Emergency Braking (AEB)application, an Adaptive Cruise Control (ACC) application, a Blind SpotDetection (BSD) application, and an autonomous driving application. If auser input on any one of the displayed icons 1881 and 1882 is received,the processor 270 can control the display unit 251 to display anapplication screen corresponding to autonomous driving.

FIG. 19 is a flowchart illustrating how a user interface apparatus for avehicle operates according to an embodiment of the present invention.Referring to FIG. 19, the processor 270 can detect a gaze of a drivervia the internal camera 220 in S1910. The gaze of a driver may indicatea point at which eyes of the driver stare, or a direction of the eyes ofthe driver. The processor 270 can control the display unit 251 based ondriver gaze information in S1920. For example, the processor 270 candetermine whether a gaze of the driver is beyond a preset range.Further, the preset range may indicate a range that is preset in orderto determine whether the gaze of the driver is toward the area forwardof the vehicle 100.

If it is determined that the gaze of the driver is beyond the presetrange, the processor 270 can control the display unit 251 to perform apreset function. The preset function may be functions for inducing thedriver to look ahead. The preset function may include a function ofdisabling displaying of a screen. For example, the preset function mayinclude a function of disabling a user's touch input. In this instance,the processor 270 can not respond to the user's touch input.

In addition, the preset function may include a function of outputting amessage for inducing the driver to look ahead, or a function ofdisplaying a vehicle front view. For example, the preset function mayinclude a function of displaying a screen, displayed on the display unit251, in an additionally provided HUD or cluster. Also, if it isdetermined that the gaze of the driver is beyond a preset range, theprocessor 270 can switch the state of the vehicle 100 from a manualdriving state to an autonomous driving state.

Next, FIG. 20 is a diagram illustrating an example of how a userinterface apparatus for a vehicle operates based on driver gazeinformation according to an embodiment of the present invention. Asshown in 2010 of FIG. 20, the processor 270 can detect a gaze of adriver using the internal camera 220 to generate driver gaze informationand control the display unit 251 based on the driver gaze information.Further, the processor 270 can determine whether a gaze 2001 of thedriver is beyond a preset range 2002.

As shown in 2020 of FIG. 20, if it is determined that the gaze 2001 ofthe driver has been beyond the preset range 2002 for a preset period oftime or more, the processor 270 can control the display unit 251 todisable displaying of a screen. In this instance, the processor 270 candisable a touch input function in a touch screen. As shown in 2030 ofFIG. 20, if it is not determined that the gaze 2001 of the driver isbeyond the preset range 2002, the processor 270 can control the displayunit 251 to display a screen normally.

FIG. 21 is a flowchart illustrating an operation of a user interfaceapparatus for a vehicle according to an embodiment of the presentinvention and FIGS. 22 to 24 are diagrams illustrating examples of theoperation described with reference to FIG. 21. Referring to FIG. 21, theprocessor 270 can determine whether the vehicle 100 is in a drivingstate in S2110. If it is not determined that the vehicle 100 is in thedriving state, the processor 270 can control the display unit 251 tooperate normally in S2190.

If it is determined that the vehicle 100 is in the driving state, theprocessor 270 can determine whether a gaze (indicated by 2001 in FIG.20) of a driver is beyond a preset range (indicated by 2002 in FIG. 20)in S2120. If it is not determined that the gaze of the driver is beyondthe preset range, the processor 270 can control the display unit 251 tooperate normally in S2190.

If it is determined that the gaze of the driver is beyond the presetrange, the processor 270 can determine whether the gaze of the driverhas been beyond the preset range for a first period of time or more inS2130. If it is determined that the gaze of the driver is has beenbeyond the preset range for less than the first period of time, theprocessor 270 can return to operation S2120.

If it is determined that the gaze of the driver has been beyond thepreset range for the first period of time or more, the processor 270 cancontrol the display unit 251 to switch a first screen into a secondscreen in S2140. The second screen may be a screen in which displayingsome of the information items included in the first screen is disabled.

The second screen may include a lesser number of information items thanthe first screen. The second screen may include information that a usercan more intuitively perceive, compared to the first screen. Forexample, the first screen may be configured with texts, and the secondscreen may be configured with an image.

Referring to FIG. 22, the processor 270 can display a first screen 2210.The first screen may include a plurality of information items 2211 and2212 or a plurality of content items. If it is determined that a gaze ofa driver has been beyond a preset range for the first period of time ormore, the processor 270 can control the display unit 251 to switch thefirst screen 2210 into a second screen 2220. Further, the second screen2220 may be a screen in which some information items 2212 are disabled,compared to the first screen 2210. In addition, the processor 270 cancontrol the display unit 251 to display a message 2211 for inducing adriver to look ahead.

Referring again to FIG. 21, the processor 270 can determine whether thegaze of the driver has been beyond the preset range for a second periodof time or more in S2150. The second period of time may be longer thanthe first period of time. If it is determined that the gaze of thedriver has been beyond the preset range for less than the second periodof time, the processor 270 can return to operation S2120.

If it is determined that the gaze of the driver has been beyond thepreset range for the second period of time or more, the processor 270can disable a touch input function in S2160. In this instance, theprocessor 270 can control the display unit 251 to display a notificationto inform that the touch input function is disabled.

Referring to FIG. 23, if it is determined that the gaze of the driverhas been beyond the present range for the second period of time or more,the processor 270 can disable the touch input function. In thisinstance, the processor 270 can control the display unit 251 to displaya notification 2310 to inform that the touch input function is disabled.

In addition, the processor 270 can change a user input means. Forexample, the processor 270 can change a type of a user input from atouch input to a voice input or a gesture input. The processor 270 canalso control the display unit 251 to display a notification 2320 toinform that inputs (e.g., a voice input and a gesture input) other thana touch input are enabled. Further, the processor 270 can control thedisplay unit 251 based on a user input that is received by use of achanged user input means.

Again, referring to FIG. 21, the processor 270 can determine whether thegaze of the driver has been beyond the preset range for a third periodof time or more in S2170. The third period of time may be longer thesecond period of time. If it is determined that the gaze of the driverhas been beyond the preset range for less than the third period of time,the processor 270 can return to operation S2120. If the gaze of thedriver is determined to have been beyond the preset range for the thirdperiod of time or more, the processor 270 can turn off the display unit251 or output a vehicle front view in S2180.

Referring to FIG. 24, if it is determined that the gaze of the driverhas been beyond the preset range for the third period of time or more,the processor 270 can turn off the display unit 251. In this instance,nothing may be displayed on the display unit 251. If it is determinedthat the gaze of the driver has been beyond the preset range for thethird period of time or more, the processor 270 can output a vehiclefront view 2410. In this instance, the processor 270 can control thedisplay unit 251 to display an image 2211 for inducing the driver tolook ahead. In addition, the vehicle front view 2410 may be an imageacquired by the camera 310.

FIGS. 25A to 27 are diagrams illustrating various examples of how a userinterface apparatus operates based on a user's gaze according to anembodiment of the present invention. In addition, the user includes adriver and a passenger sitting on the front passenger seat.

As illustrated in FIG. 25A, a user interface apparatus 200 may include aplurality of display units. For example, the user interface apparatus200 may include a first display unit 251 b and a second display unit 251e. The first display unit 251 b can be disposed close to a driver,compared to the second display unit 251 e.

The processor 270 can control the plurality of display units 251 b and251 e individually based on driver gaze information. For example, theprocessor 270 cancan perform a control action such that if the period oftime for which driver gazes out of the predetermined range is long, ascreen being displayed on a display unit (the display unit 251 e) moredistant from the driver among the plurality of display units maydisappear. The feature of causing a screen to disappear may includeturning off the display unit 251, entering a standby mode, or convertingthe currently displayed screen into a different screen.

For example, if it is determined that the gaze 2001 of the driver hasbeen beyond the preset range 2002 for a first period of time or more,the processor 270 can cause a screen displayed on the second displayunit 251 e to disappear. Then, if it is determined that the gaze 2001 ofthe driver has been beyond the preset range 2002 for a second period oftime or more, the processor 270 can cause screens displayed on the firstand second display units 251 b and 251 e to disappear. The second periodof time may be longer than the first period of time. In this way, adriver may spend less time in staring at a display unit disposed distantfrom the driver among a plurality of display units, so that the drivermay be induced to focus on driving.

As illustrated in FIG. 25B, the processor 270 can detect a gaze of adriver 2510 and a gaze of a person sitting in the front passenger seat(hereinafter, referred to as a passenger) (2520). If it is determinedthat a gaze 2001 a of the driver and a gaze 2001 b of the passenger havebeen directed toward the display unit 251 for a preset period of time ormore, the processor 270 can output a notification message 2530 to inducethe driver 2510 to stare at a preset range 2002 a.

As illustrated in FIG. 25C, if it is determined that the gaze 2001 a ofthe driver and the gaze 2001 b of the passenger have been directedtoward the display unit 251 for a preset period of time or more, theprocessor 270 can change a display format of a screen being displayed onthe display unit 251, so that the passenger 2520, rather than the driver2510, may take control of the display unit 251. For example, theprocessor 270 can divide the display unit 251 into two regions 2541 and2542 and control the display unit 251 so that the currently displayedscreen is output in any one of the two regions 2541 and 2542.

The region 2541 in which the screen is output may be a region close tothe front passenger seat. In this instance, the region 2542 in which thescreen is not output may output a message for inducing the driver tolook ahead. Alternatively, the processor 270 can output a vehicle frontview in the region 2542 in which the screen is not output.

In addition, the processor 270 can maintain the touch input function inthe region 2541 in which the screen is output. The processor 270 candisable the touch input function in the region 2542 in which the screenis not output. If it is determined that the gaze 2001 of the driverfalls within the preset range 2002 a, the processor 270 can control thedisplay unit 251 to restores the original display format of the screen.

As illustrated in FIG. 25D, if it is determined that the gaze 2001 a ofthe driver and the gaze 2001 b of the passenger have been directedtoward the display unit 251 for a preset period of time or more, theprocessor 270 can output a message for inducing the driver to lookahead. Alternatively, the processor 270 can control the display unit 251to output a vehicle front view.

FIG. 26 is a diagram illustrating how a user interface apparatus for avehicle disables selection of an item having a different depth accordingto an embodiment of the present invention. Referring to FIG. 26, theprocessor 270 can control the display unit 251 to output a plurality ofitems (e.g., graphic objects, icons, widgets, buttons, and images) forselection. Each of the items may be formed to have a preset depth level.

A depth level may be defined as the number of user inputs that arerequired for execution of a function associated with a correspondingitem. For example, a 1-depth item 2610 may be an item that requires asingle user input to execute a function associated therewith. Forexample, a 2-depth item 2620 may be an item that requires double userinputs to execute a function associated therewith. The higher depthlevel an item has, the more number of user inputs a user needs to applyto execute a desired function. Accordingly, if a driver behind the wheelwants to execute a function of an item having a higher depth, the driverneeds a long time for the execution while not looking ahead.

If the time for which a gaze of the driver has been beyond a presetrange becomes longer, the processor 270 can disable selection of menuitems sequentially in a descending order of depth levels thereof. Forexample, if it is determined that the gaze of the driver has been beyondthe preset range for a first period of time or more, the processor 270can disable selection of the 3-depth item 2630.

For example, if it is determined that the gaze of the driver has beenbeyond the preset range for a second period of time or more, theprocessor 270 can disable selection of the 2-depth item 2620 and the3-depth item 2630. For example, if it is determined that the gaze of thedriver has been beyond the preset range for a third period of time ormore, the processor 270 can disable selection of the 1-depth to 3-depthitems 2610, 2620, and 2630. In this instance, the processor 270 cancontrol the display unit 251 to output a message for inducing the driverto look ahead.

FIG. 27 is a diagram illustrating how a user interface apparatus for avehicle disables a touch input function according to an embodiment ofthe present invention. Referring to FIG. 27, if it is determined that agaze 2001 of a driver has been beyond a preset range for a certainperiod of time or more, the processor 270 can disable an input functionin the display unit 251. In this instance, the processor 270 can controlthe display unit 251 to output information 2710 about remaining timeuntil the input function is disabled.

FIG. 28 is a diagram illustrating how a user interface apparatus for avehicle, including a plurality of display units, operates according toan embodiment of the present invention. Referring to FIG. 28, a userinterface apparatus 200 may include a plurality of display units 251 a,251 b, 251 c, and 251 d. For example, the user interface apparatus 200may include a first display unit 251 a, a second display unit 251 b, athird display unit 251 c, and a fourth display unit 251 d.

For example, the first display unit 251 a may be a display that operatesas a cluster. For example, the second display unit 251 may be a CenterInformation Display (CID) disposed on the center fascia. A plurality ofscreens may be displayed on the second display unit 251 b. A pluralityof information items may be displayed on the second display unit 251 b.For example, the third display unit 251 c may be a Head Up Display(HUD), and the fourth display unit 251 d may be a Rear SeatEntertainment (RSE) display.

If a user input is received when a plurality of screens are displayed onthe second display unit 251 b, the processor 270 can control theplurality of screens to be separately displayed on the different displayunits 251 a, 251 c, and 251 d. For example, if a user input is receivedwhen a plurality of screens are displayed on the second display unit 251b, the processor 270 cancan perform a control action such that a firstscreen is displayed on the first display unit 251 a, a second screen isdisplayed on the second display unit 251 b, a third screen is displayedon the third display unit 251 c, and a fourth screen is displayed on thefourth display unit 251 d.

FIG. 29 is a diagram illustrating how a user interface apparatus for avehicle, including a plurality of display units, operates according toan embodiment of the present invention. Referring to FIG. 29, a userinterface apparatus 200 may include a first display unit 251 b and asecond display unit 251 c.

For example, the first display unit 251 b may be a CID disposed in thecenter fascia, and the second display unit 251 c may be an HUD. Theprocessor 270 can control the second display unit 251 c based on a touchinput received on the first display unit 251 b. If a drag input isreceived in the first display unit 251 b when a media file controlscreen is displayed on the second display unit 251 c, the processor 270can play a next file. Also, if a pinch-in input is received in the firstdisplay unit 251 b when a navigation screen is displayed on the seconddisplay unit 251 c, the processor 270 can reduce the size of thenavigation screen.

FIG. 30 is a diagram illustrating how a user interface apparatus for avehicle used by multiple users operates according to an embodiment ofthe present invention. Referring to FIG. 30, the vehicle 100 may be usedby multiple users. Information on each of the multiple users may bestored in the memory 240. The memory 240 may store information on eachof multiple users 3010 and 3020. Information on a user may includeauthentication information used to authenticate the user, and GraphicUser Interface (GUI) information of the display unit 251, which is setby the user. The authentication information may include biometricinformation of the user.

The processor 270 can set the display unit 251 based on GUI informationcorresponding to an authenticated user. For example, if a first user3010 is in the vehicle 100, the processor 270 can acquire biometricinformation of the first user 3010 using the biometric sensing unit 230,and compare the biometric information of the first user 2010 withauthentication information stored in the memory 240 to authenticate thefirst user 3010. If the first user 3010 is authenticated, the processor270 can set the display unit 251 according to GUI set by the first user3010. For example, if a second user 3020 is in the vehicle 100, theprocessor 270 can set the display unit 251 according to pre-stored GUIcorresponding to the second user 3020.

FIGS. 31 to 33 are diagrams illustrating how to switch to an autonomousdriving state by a user interface apparatus for a vehicle according toan embodiment of the present invention. Referring to FIG. 31, if apreset application is executed, the processor 270 can provide a signalto switch the state of the vehicle 100 from a manual driving state to anautonomous driving state. For example, if a touch input on an icon 3110is received when an icon 3110 corresponding to an autonomous drivingapplication is displayed, the processor 270 can provide a signal so thatthe state of the vehicle 100 is switched from the manual driving stateto the autonomous driving state.

Referring to FIG. 32, while a preset application is executed, theprocessor 270 can enter a preset depth level or higher of theapplication. In this instance, the processor 270 can provide a signal toswitch the state of the vehicle 100 from the manual driving state to theautonomous driving state. For example, the processor 270 can execute afirst application in accordance with a first user input 3210. Theexecution state of the first application may be defined as 1-depth.While the first application is executed, the processor 270 can enter 2depth in accordance with a second user input 3220.

Entering 2 depth or a higher level of a specific application may be setas a criteria for switching to an autonomous driving state. Whenentering 2 depth of the specific application, the processor 270 canprovide a signal so that the state of the vehicle 100 is switched fromthe manual driving state to the autonomous driving state.

Referring to FIG. 33, the processor 270 can detect a gaze of a driverusing the external camera 220, and generate driver gaze information. Theprocessor 270 can determine whether a gaze 2001 of the driver has beenbeyond a preset region 2002 for a preset period of time or more. If itis determined that the gaze 2001 of the driver has been beyond thepreset region 2002 for the preset period of time or more, the processor270 can provide a signal to switch the state of the vehicle 100 from amanual driving state to an autonomous driving state.

For example, the processor 270 can determine whether the gaze 2001 ofthe driver has been directed toward the display unit 251 for a presetperiod of time or more. If it is determined that the gaze 2001 of thedriver has been directed toward the display unit 251 for the presetperiod of time or more, the processor 270 can provide a signal to switchthe state of the vehicle 100 from the manual driving state to theautonomous driving state.

FIGS. 34 to 36 are diagrams illustrating how to operate after switch toan autonomous driving state according to an embodiment of the presentinvention. Referring to the drawings, if criteria for switching to anautonomous driving state is satisfied, the processor 270 can control thedisplay unit 251 to display a screen 3410 for autonomous drivingoperations. The criteria for switching to an autonomous driving statemay be based on at least one of the following: information on executionof a preset application, information about entrance in a preset level orhigher of a preset application, driver gaze information, information ona degree of driving difficulty, speed information, and traffic flowinformation.

The screen corresponding to autonomous driving may include a contentprovision screen 3410, as shown in FIG. 34. For example, the screencorresponding to autonomous driving may include at least one of a screenof providing image contents and a screen of providing music contents.The screen corresponding to autonomous driving may include a screen foroperation of a vehicle utility device. For example, the screencorresponding to autonomous driving may include at least one of thefollowing: a screen for operation of a navigation device, a screen foroperation of a communication device, a screen for operation of a musicplay device, a screen for operation of a schedule management device, anda screen for operation of a video play device. In addition, theprocessor 270 can receive emergency situation information from theobject detection device 300 via the interface 245.

As illustrated in FIG. 35, the object detection device 300 may generateemergency situation information based on TTC between the vehicle 100 andan object 3510 (e.g., a preceding object) which is located outside thevehicle 100. The emergency situation information may be generated basedon the information on the TTC between the vehicle 100 and the object3510.

The object detection device 300 may generate emergency situationinformation based on information on a distance between the vehicle 100and the object 3510 outside the vehicle 100, and information on a speedof the vehicle 100 relative to the object 3510 outside the vehicle 100.Emergency situation information may be generated based on theinformation on the distance between the vehicle 100 and the object 3510outside the vehicle 100, and the information on the speed of the vehicle100 relative to the object 3510 outside the vehicle 100. For example,the emergency situation information may be information about a predictedcollision between the vehicle 100 and the object 3510.

As illustrated in FIG. 36, if emergency situation information isreceived via the interface 245, the processor 270 can display a screencorresponding to autonomous driving. The screen corresponding toautonomous driving may include a brake input button 3610 or a steeringinput button 3620. Further, the processor 270 can control the displayunit 251 to display the brake input button 3610 or the steering inputbutton 3620.

If a user input on the brake input button 3610 is received, theprocessor 270 can provide a signal to the vehicle drive device 600 tocontrol operation of a brake apparatus. Even when the user input isreceived from a passenger, the processor 270 can provide a signal tocontrol operation of the brake apparatus.

If the user input on the brake input button 3610 is received after agesture is determined to be applied from the front passenger seat, theprocessor 270 can provide a signal to control operation of the brakeapparatus. If a user input on the steering input button 3620 isreceived, the processor 270 can provide a signal to the vehicle drivedevice 600 to control operation of a steering apparatus.

Even when the user input is received from a passenger, the processor 270can provide the signal to control operation of the steering apparatus.If the user input on the steering input button 3620 is received after agesture is determined to be applied from the front passenger seat, theprocessor 270 can provide a signal to control operation of the steeringapparatus.

Embodiments of the present invention have one or more of the followingeffects. First, by controlling an output differently based on whether agesture is applied from the driver seat or the front passenger seat, itis possible to help a driver to focus on driving and therefore preventan accident. Second, when a driver does not focus on driving, it ispossible to forcibly switch to an autonomous driving mode for thepurpose of safety. Third, an output of a display is controlled based onvehicle state information or driver gaze information, thereby reducing apossibility for an occident to occur.

The present invention as described above may be implemented as code thatcan be written on a computer-readable medium in which a program isrecorded and thus read by a computer. The computer-readable mediumincludes all kinds of recording devices in which data is stored in acomputer-readable manner. Examples of the computer-readable recordingmedium may include a hard disk drive (HDD), a solid state disk (SSD), asilicon disk drive (SDD), a read only memory (ROM), a random accessmemory (RAM), a compact disk read only memory (CD-ROM), a magnetic tape,a floppy disc, and an optical data storage device. In addition, thecomputer-readable medium may be implemented as a carrier wave (e.g.,data transmission over the Internet). In addition, the computer mayinclude a processor or a controller. Thus, the above detaileddescription should not be construed as being limited to the embodimentsset forth herein in all terms, but should be considered by way ofexample. The scope of the present invention should be determined by thereasonable interpretation of the accompanying claims and all changes inthe equivalent range of the present invention are intended to beincluded in the scope of the present invention.

Although embodiments have been described with reference to a number ofillustrative embodiments thereof, it should be understood that numerousother modifications and embodiments can be devised by those skilled inthe art that will fall within the spirit and scope of the principles ofthis disclosure. More particularly, various variations and modificationsare possible in the component parts and/or arrangements of the subjectcombination arrangement within the scope of the disclosure, the drawingsand the appended claims. In addition to variations and modifications inthe component parts and/or arrangements, alternatives uses will also beapparent to those skilled in the art.

What is claimed is:
 1. A user interface apparatus configured to beinstalled in a vehicle, the apparatus comprising: a touch screen; agesture detecting unit configured to detect a gesture of a user andconvert the gesture into an electrical signal; and at least oneprocessor configured to: determine whether a criteria for switching toan autonomous driving state is satisfied in response to the gesturedetected by the gesture detecting unit being applied from a driver seatof the vehicle to the touch screen, wherein, in response to the criteriafor switching to the autonomous driving state being satisfied, the atleast one processor is further configured to switch a state of thevehicle to the autonomous driving state.
 2. The user interface apparatusaccording to claim 1, wherein, in response to execution of a presetapplication, the at least one processor is further configured to switchthe state of the vehicle to the autonomous driving state, or wherein,when entering a preset depth level of a preset application in responseto a user input while the preset application is being executed, the atleast one processor is further configured to switch the state of thevehicle to the autonomous driving state.
 3. The user interface apparatusaccording to claim 1, wherein the at least one processor is furtherconfigured to: display a driver screen on the touch screen correspondingto driver operations of the vehicle in response to the gesture detectedby the gesture detecting unit being applied from a driver seat of thevehicle to the touch screen; and display a passenger screen on the touchscreen corresponding to passenger operations of the vehicle in responseto the gesture detected by the gesture detecting unit being applied froma front passenger seat of the vehicle to the touch screen.
 4. Anautonomous vehicle comprising: a user interface apparatus installed inthe autonomous vehicle, wherein the user interface apparatus comprises:a touch screen; a gesture detecting unit configured to detect a gestureof a user and convert the gesture into an electrical signal; and atleast one processor configured to: determine whether a criteria forswitching to an autonomous driving state is satisfied in response to thegesture detected by the gesture detecting unit being applied from adriver seat of the vehicle to the touch screen, wherein, in response tothe criteria for switching to the autonomous driving state beingsatisfied, the at least one processor is further configured to switch astate of the vehicle to the autonomous driving state.
 5. The autonomousvehicle according to claim 4, wherein, in response to execution of apreset application, the at least one processor is further configured toswitch the state of the vehicle to the autonomous driving state, orwherein, when entering a preset depth level of a preset application inresponse to a user input while the preset application is being executed,the at least one processor is further configured to switch the state ofthe vehicle to the autonomous driving state.
 6. The autonomous vehicleaccording to claim 4, wherein the at least one processor is furtherconfigured to: display a driver screen on the touch screen correspondingto driver operations of the vehicle in response to the gesture detectedby the gesture detecting unit being applied from a driver seat of thevehicle to the touch screen; and display a passenger screen on the touchscreen corresponding to passenger operations of the vehicle in responseto the gesture detected by the gesture detecting unit being applied froma front passenger seat of the vehicle to the touch screen.
 7. A methodof controlling a vehicle, comprising: detecting, by a gesture detectingunit, a gesture of a user and converting the gesture into an electricalsignal; determining, by at least one processor, whether a criteria forswitching to an autonomous driving state is satisfied in response to thegesture detected by the gesture detecting unit being applied from adriver seat of the vehicle to the touch screen; and switching, by the atleast one processor, a state of the vehicle to the autonomous drivingstate in response to the criteria for switching to the autonomousdriving state being satisfied.
 8. The method according to claim 7,further comprising: switching, by the at least one processor, the stateof the vehicle to the autonomous driving state in response to executionof a preset application.
 9. The method according to claim 7, furthercomprising: switching, by the at least one processor, the state of thevehicle to the autonomous driving state when entering a preset depthlevel of a preset application in response to a user input while thepreset application is being executed.
 10. The method according to claim7, further comprising: displaying, by the at least one processor, adriver screen on the touch screen corresponding to driver operations ofthe vehicle in response to the gesture detected by the gesture detectingunit being applied from a driver seat of the vehicle to the touchscreen; and displaying, by the at least one processor, a passengerscreen on the touch screen corresponding to passenger operations of thevehicle in response to the gesture detected by the gesture detectingunit being applied from a front passenger seat of the vehicle to thetouch screen.